0.1 Data Description Three types of data were recorded in London between November 2011 and February 2014
0.2 Weather Data Description Features recorded in 1 hour resolution:
0.3 Introducing Temporal Features
Extracted temporal features:
Encoding features on unit circle:
1.1 Selecting a Building: for 1 household, in half an hour resolution. We look for a building that has:
searching block 77 Best in the block: house id MAC000068 with maximum 17 consecutive zeros
1.2 Cleaning Data
Data starts at
2011-12-09
and ends at
2014-02-28
Only considering data collected from
2012-01-01
to
2013-12-31
Data has missing row in
2012-10-25
[22.]
2013-01-30
[1.5]
2013-04-22
[12.5]
2013-07-03
[10.5]
2013-08-02
[19.5]
2013-11-18
[12. 12.5 13. 13.5 14. 14.5 15. 15.5]
Added row for missing time steps
Duplicate rows:
energy date hourofd
16927 NaN 2012-12-18 15.0
All days have 48 records
Data contains 13 NaNs in columns
['energy']
Replaced by interpolation.
1.3 Add Temporal Factors
number of holidays 25.0
1.4 Add Weather Features
Check for missing dates in the hourly data:
Data has missing row in
2013-09-09
[23]
2013-09-10
[0]
Added row for missing time steps
Hourly weather contains 4 NaNs in rows:
temperature icon date hourofd
14831 NaN NaN 2013-09-09 23
14832 NaN NaN 2013-09-10 0
Hourly weather nans were interpolated
Missing dates of daily icons:
[datetime.date(2012, 4, 28), datetime.date(2012, 5, 3), datetime.date(2012, 10, 12), datetime.date(2012, 10, 19), datetime.date(2012, 10, 23), datetime.date(2012, 10, 26), datetime.date(2013, 4, 26), datetime.date(2013, 5, 21), datetime.date(2013, 8, 24), datetime.date(2013, 9, 6), datetime.date(2013, 9, 9)]
energy date hourofd hourofd_x hourofd_y weekday flag_weekend \
0 0.572 2012-01-01 0.0 0.000000 1.000000 Sunday True
1 0.910 2012-01-01 0.5 0.130526 0.991445 Sunday True
2 0.142 2012-01-01 1.0 0.258819 0.965926 Sunday True
3 0.696 2012-01-01 1.5 0.382683 0.923880 Sunday True
4 0.392 2012-01-01 2.0 0.500000 0.866025 Sunday True
dayofy dayofy_x dayofy_y daypart month season flg_holiday day_type \
0 0.0 0.0 1.0 night 1 winter False weekend
1 0.0 0.0 1.0 night 1 winter False weekend
2 0.0 0.0 1.0 night 1 winter False weekend
3 0.0 0.0 1.0 night 1 winter False weekend
4 0.0 0.0 1.0 night 1 winter False weekend
temperature_hourly icon_hourly icon_daily
0 12.12 partly-cloudy-night partly-cloudy-day
1 12.12 partly-cloudy-night partly-cloudy-day
2 12.59 cloudy partly-cloudy-day
3 12.59 cloudy partly-cloudy-day
4 12.45 partly-cloudy-night partly-cloudy-day
energy date hourofd hourofd_x hourofd_y weekday \
35083 1.270 2013-12-31 21.5 -0.608761 0.793353 Tuesday
35084 0.822 2013-12-31 22.0 -0.500000 0.866025 Tuesday
35085 0.626 2013-12-31 22.5 -0.382683 0.923880 Tuesday
35086 0.726 2013-12-31 23.0 -0.258819 0.965926 Tuesday
35087 0.630 2013-12-31 23.5 -0.130526 0.991445 Tuesday
flag_weekend dayofy dayofy_x dayofy_y daypart month season \
35083 False 364.0 -0.017213 0.999852 evening 12 winter
35084 False 364.0 -0.017213 0.999852 night 12 winter
35085 False 364.0 -0.017213 0.999852 night 12 winter
35086 False 364.0 -0.017213 0.999852 night 12 winter
35087 False 364.0 -0.017213 0.999852 night 12 winter
flg_holiday day_type temperature_hourly icon_hourly icon_daily
35083 False weekday 5.92 clear-night partly-cloudy-day
35084 False weekday 6.54 clear-night partly-cloudy-day
35085 False weekday 6.54 clear-night partly-cloudy-day
35086 False weekday 7.43 clear-night partly-cloudy-day
35087 False weekday 7.43 clear-night partly-cloudy-day
2.1 Plot monthly patterns